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Abstract 

The Libra Toolkit is a collection of algorithms for learning and inference with discrete proba¬ 
bilistic models, including Bayesian networks, Markov networks, dependency networks, and 
sum-product networks. Compared to other toolkits, Libra places a greater emphasis on 
learning the structure of tractable models in which exact inference is efficient. It also in¬ 
cludes a variety of algorithms for learning graphical models in which inference is potentially 
intractable, and for performing exact and approximate inference. Libra is released under a 
2-clause BSD license to encourage broad use in academia and industry. 

Keywords: Probabilistic graphical models, structure learning, inference 

1. Introduction 

The Libra Toolkit is a collection of algorithms for learning and inference with probabilistic 
models in discrete domains. What distinguishes Libra from other toolkits is the types 
of methods and models it supports. Libra includes a number of algorithms for structure 
learning for tractable probabilistic models in which exact inference can be done efficiently. 
Such models include sum-product networks (SPN), mixtures of trees (MT), and Bayesian 
and Markov networks with compact arithmetic circuits (AC). These learning algorithms are 
not available in any other open-source toolkit. Libra also supports strueture learning for 
graphical models, such as Bayesian networks (BN), Markov networks (MN), and dependency 
networks (DN), in which inference is not necessarily tractable. Some of these methods are 
unique to Libra as well, such as using dependency networks to learn Markov networks. Libra 
provides a variety of exact and approximate inference algorithms for answering probabilistic 
queries in learned or manually specified models. Many of these are designed to exploit local 
structure, such as conjunctive feature functions or tree-structured conditional probability 
distributions. 

The overall goal of Libra is to make these methods available to researchers, practition¬ 
ers, and students for use in experiments, applications, and education. Each algorithm in 
Libra is implemented in a command-line program suitable for interactive use or scripting, 
with consistent options and file formats throughout the toolkit. Libra also supports the de- 
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Learning General Models 

o BN structure with tree CPDs i 

DN structure with tree/boosted tree/LR GPDs i 

• MN structure from DNs ( 

MN parameters (pseudo-likelihood) 

Chickering et al. 

1997 

■Heckerman et al. 

2009 1 
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Learning Tractable Models 


Tractable BN/AC structure 
Tractable MN/AC structure 
Mixture of trees (MT) 

SPN structure (ID-SPN algorithm) 

Chow-Liu algorithm 

AC parameters (maximum likelihood) 


iLowd and Domingos |2Q08 i 


( ■Lowd and Hoosiienasr I 

(Meila and Jordan ' TDC ^ 


IKooshenas and Lowd 


(Chow and Liu ' 19b8|l 
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Approximate Inference 


Gibbs sampling (BN,MN,#DN) 

Mean field {BN,MN,*DN) 

Loopy belief propagation (BN,MN) 
Max-product (BN,MN) 

Iterated conditional modes (BN,MN,*DN) 
Variational optimization of ACs 


( Heckerman et al. 2000 i (DN) 

(Lowd and bhama^ ' 5UlT \ (DN) 


(Lowd and Domingos 20101 


Exact Inference 

o AC variable elimination (BN,MN) 
o Marginal and MAP inference (AC,SPN,MT) 

Chavira and Darwiche 20071 

Darwiche 

2DD3 


Table 1: Learning and inference algorithms implemented in Libra. Filled circles (•) indicate 
algorithms that are unique to Libra, and hollow circles (o) indicate algorithms with 
no other open-source implementation. 


velopment of new algorithms through modular code organization, including shared libraries 
for different representations and file formats. 

Libra is available under a modified (2-clause) BSD license, which allows modification 
and reuse in both academia and industry. 

2. Functionality 

Libra includes a variety of learning and inference algorithms, many of which are not available 
in any other open-source toolkit. See Table for a brief overview. 

Libra’s command-line syntax is designed to be simple. For example, to learn a tractable 
BN, run the command: “libra acbn -i train, data -mo model, bn -o model. ac” where 
train.data is the input data, model.bn is the filename for saving the learned BN, and 
model. ac is the filename for the corresponding AC representation, which allows for ef¬ 
ficient, exact inference. To compute exact conditional marginals in the learned model: 
“libra acquery -m model, ac -ev test.ev -marg”. To compute approximate marginals 
in the BN with loopy belief propagation: “libra bp -m model.bn -ev test.ev”. Addi¬ 
tional command-line parameters can be used to specify other options, such as the priors 
and heuristics used by acbn or the maximum number of iterations for bp. These are just 
three of more than twenty commands included in Libra. 

Libra supports a variety of file formats. For data instances, Libra uses comma separated 
values, where each value is a zero-based index indicating the discrete value of the corre¬ 
sponding variable. For evidence and query files, unknown or missing values are represented 
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Representation 

Inference 

Learning 

Toolkit 

Model Types 

Factors 

Exact 

Approx. 

Param. 

Structure 

Libra 

BN,MN,DN,SPN,AC 

Tree, Feature 

ACVE 

G,BP,MF 

ML,PL 

BN,...,AC 

Fastinf 

BN,MN 

Table 

JT 

Many 

ML,EM 

- 

libDAI 

BN,MN 

Table 

JT,E 

Many 

ML,EM 

- 

OpenGM2 

BN,MN 

Sparse 

- 

Many 

- 

- 

Banjo 

BN,DBN 

Table 

- 

- 

- 

BN 

BNT 

BN,DBN,ID 

LR,OR,NN 

JT,VE,E 

G,LW,BP 

ML,EM 

BN 

Deal 

BN 

Table 

- 

- 

- 

BN 

OpenMarkov 

BN,MN,ID 

Tree,ADD,OR 

JT 

LW 

ML 

BN,MN 

SMILE 

BN,DBN,ID 

Table 

JT 

Sampling 

ML,EM 

BN 

UnB Bayes 

BN,ID 

Table 

JT 

G,LW 

- 

BN 


Table 2: Comparison of Libra to several other probabilistic inference and learning toolkits. 


with the special value . For model files, Libra supports the XMOD representation from 
the WinMine Toolkit, the Bayesian interchange format (BIF), and the simple representation 
from the UAI inference competition. Libra converts among these different formats using 
the provided mconvert utility, as well as to its own internal formats for BNs, MNs, and 
DNs (.bn, .mn, .dn). Libra has additional representations for ACs and SPNs (.ac, .spn). 
These formats are designed to be easy for humans to read and programs to parse. 

Libra is implemented in OCaml. OCaml is a statically typed language that supports 
functional and imperative programming styles, compiles to native machine code on multiple 
platforms, and uses type inference and garbage collection to reduce programmer errors and 
effort. OCaml has a good foreign function interface, which Libra uses for linking to C 
libraries and implementing some memory-intensive subroutines more efficiently in C. The 
code to Libra includes nine support libraries, which provide modules for input, output, and 
representation of different types of models, as well as commonly used algorithms and utility 
methods. 


3. Comparison to Other Toolkits 


In Table we compare Libra to other toolkits in terms of representation, learning, and 
inference. 

In terms of representation, Libra is the only open-source software package that supports 
ACs and one of a very small number that support DNs or SPNs. Libra does not currently 
support dynamic Bayesian networks (DBN) or influence diagrams (ID). For factors, Libra 
supports tables, trees, and arbitrary conjunctive feature functions. BNT (Murphy, 2001) 


and OpenMarkov (CISIAD, 2013) also support additional types of CPDs, such as logistic 
regression, noisy-OR, neural networks, and algebraic decision diagrams, but they only sup¬ 


port tabular CPDs for structure learning. OpenGM2 (Andres et al. 2012) supports sparse 


factors, but iterates through all factor states during inference. Libra is unique in its ability 
to learn models with local structure and exploit that structure in inference. 

For exact inference, the most common algorithms are junction tree (JT), enumeration 


(E), and variable elimination (VE). Libra provides ACVE (Chavira and Darwiche, 2007), 


which is similar to building a junction tree, but it can exploit structured factors to run 
inference in many high-treewidth models. Eor approximate inference, Libra provides Gibbs 
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Figure 1: Running time of belief propagation and Gibbs sampling in Libra and libDAI, 
evaluated on grid-structured MNs of various sizes. 


sampling (G), loopy belief propagation (BP), and mean field (MF), all of which are op¬ 
timized for structured factors. A few learning toolkits offer likelihood weighting (LW) or 
additional sampling algorithms for BNs. Fastinf (Jaimovich et al. 2010), libDAI (Mooij[ 


2010), and OpenGM2 offer the most algorithms but only support tables. 


For learning, Libra supports maximum likelihood (ML) parameter learning for BNs, 
AGs, and SPNs, and pseudo-likelihood (PL) optimization for MNs and DNs. Libra does 
not yet support expectation maximization (EM) for learning with missing values. Structure 
learning is one of Libra’s greatest strengths. Most toolkits only provide algorithms for 
learning BNs with tabular CPDs or MNs using the PG algorithm (Spirtes et al., 1993). Libra 
includes methods for learning BNs, MNs, DNs, SPNs, and AGs, and all of its algorithms 
support learning with local structure. 

In experiments on grid-structured MNs, Libra’s implementations of BP and Gibbs sam¬ 
pling were at least as fast as libDAI, a popular C-|—I- implementation of many inference 
algorithms. The accuracy of both toolkits was equivalent. Parameter settings, such as the 
number of iterations, were identical. See Figure for more details. 


4. Conclusion 

The Libra Toolkit provides algorithms for learning and inference in a variety of probabilistic 
models, including BNs, MNs, DNs, SPNs, and AGs. Many of these algorithms are not 
available in any other open-source software. Libra’s greatest strength is its support for 
tractable probabilistic models, for which very little other software exists. Libra makes it 
easy to use these state-of-the-art methods in experiments and applications, which we hope 
will accelerate the development and deployment of probabilistic methods. 
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